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(54) Clip display method and display device therefor 



(57) The present invention aims to provide a clip dis- 
play method Whereby the contents of a clip are clearly 
displayed and can be used to facilitate the editing of a 
motion picture. According to a method of the present in- 
vention, whereby clips obtained by dividing a motion pic- 
ture are displayed by employing typical images of indi- 
vidual clips, a graphic representation 24 of the picture 



images for a motion picture is divided to provide seg- 
ments corresponding to the ratio of the lengths of the 
clips to the overall length of the motion picture, and 
shown are sequences of the separate graphic segments 
linked to typical images. The positions of the clips rela- 
tive to the motion picture and the ratios of lengths of the 
individual clips to the overall length of the motion picture 
can be visually ascertained. 
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Description 

[0001] The present invention relates to a method 
whereby when editing a motion picture, baded from a 
disk or a video camera, individual pieces of information 
of scenes can be displayed on an editing device, and to 
a display device for this purpose; and in particular, to a 
method for displaying material to be edited that facili- 
tates the editing operation. 

[0002] Recently, the use of video devices, such as vid- 
eo cameras, that handle digital images has spread to 
the home. The editing of the images obtained with a dig- 
ital camera is an easy task, and highly individualized sto- 
ries can be prepared by employing devices that edit 
such motion pictures and change scene order while link- 
ing together only selected scenes. 
[0003] A group of images for a scene, one of a number 
of image groups obtained by dividing a series of images 
at predetermined time intervals, is called a clip. Atypical 
image (index) for each clip can be displayed on an ed- 
iting device, and an editor can then edit scenes while 
viewing these images. However, since the length of a 
clip for a scene can not be ascertained only from the 
index, editing is sometimes difficult. 
[0004] To resolve this problem, disclosed in Japanese 
Unexamined Patent Publication No. Hei 7-79404 is a 
method, as is shown in Fig. 18, whereby the indexes of 
clips, which are expressed as three-dimensional rectan- 
gular parallelepipeds, are displayed on the front faces 
of the parallelepipeds, and the lengths of the clips are 
represented by the depths of the rectangular parallele- 
pipeds. 

[0005] Disclosed in Japanese Unexamined Patent 
Publication No. Hei 7-79404 is a method, as is shown 
in Fig. 19, for reducing the indexes for clips in conso- 
nance with their lengths, and for displaying the resultant 
clips. 

[0006] A conventional display device to provide such 
a display comprises, as is shown in Fig. 20, video read- 
ing means 1 , for reading digital images supplied by a 
video camera or a disk; clip division means 2, for dividing 
a series of digital images into clips; clip division data 
storage means 4, for maintaining IN point and OUT point 
data for the obtained clips; index generation means 3, 
for generating indexes for the clips; clip length ratio cal- 
culation means 5, for calculating the ratios for the 
lengths of the individual clips; and index display means 
6, for displaying indexes by employing data received 
from the index generation means 3 and the clip length 
ratio calculation means 5. 

[0007] In this display device, when the video reading 
means 1 reads digital ' images from a video camera or 
a disk, the clip division means 2 divides the digital im- 
ages into clips. The index generation means 3 gener- 
ates typical images, which are used as indexes, using 
the first frames of the individual obtained dips and out- 
puts the stationary images to the index display means 6. 
[0008] The IN point and OUT point data for the clips 



are stored in the clip division data storage means 4. The 
clip length ratio calculation means 5 calculates the ratios 
for the lengths of the clips from the IN point and OUT 
point data, and outputs the ratio to the index display 
s means 6. Upon receipt of the data from the index gen- 
eration means 3 and the clip length ratio calculation 
means 5, the index display means 6 displays the index- 
es for the clips and the ratios for the lengths of the clips 
using the form shown in Fig. 18 or19. 
10 [0009] Using the display method in Fig. 18, however, 
it is difficult to visualize how clips are distributed and 
where they are located in a motion picture by using the 
ratios of the lengths of the clips to the total length of the 
movie. And according to the display method in Fig. 19, 
?5 since in this case the size of an index is reduced, viewing 
a typical image is difficult. 

[001 0] In order to make the editing of motion pictures 
easy, it is important that there be displayed on an editing 
screen not only the index and the length of a clip, but 

^0 also the contents of the images included in the clip, so 
that an individual performing the editing can easily un- 
derstand this data and can select appropriate scenes 
for inclusion in the clip. According to the conventional 
display method, however, no consideration of such mat- 

2S ters is taken. 

[0011] To resolve the above described conventional 
problems, it is one aim of the present invention to pro- 
vide a clip display method whereby the contents of a clip 
are clearly displayed and an be used to facilitate the ed- 

30 King of a motion picture, and to provide a display device 
therefor. 

[0012] According to a method of the present inven- 
tion, whereby clips obtained by dividing a motion picture 
are displayed by employing typical images of individual 

35 clips, a graphic representation for a motion picture is di- 
vided to provide segments corresponding to the ratio of 
the lengths of the clips to the total length of the motion 
picture, and presented are sequences of the separate 
graphic segments linked to typical images. 

40 [0013] A motion vector and a color change for a clip 
or a moving object in a motion picture is displayed in 
each corresponding segment of the graphic represen- 
tation covering the overall length of the motion picture. 
[001 4] Therefore, the locations and the lengths of the 

45 individual clips, as they are related to the motion picture, 
or the contents of the images included in the clips, can 
be ascertained at a glance, and the editing can be per- 
formed very smoothly. 

[0015] According to the present invention, a clip dis- 
ss play device comprises: index display means, for display- 
ing typical images of individual clips; clip length ratio dis- 
play means, for dividing a graphic representation of the 
overall length of a motion picture into segments at the 
ratios of the lengths of the clips to the overall length of 
« the motion picture; and linking means, for linking the typ- 
ical images displayed by the index display means with 
the segments of the graphic representation displayed 
by the clip length ratio display means. 
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[0016] The display device further comprises means 
for calculating, at a constant time interval, a motion vec- 
tor from a motion picture; color data calculation means, 
for calculating at a constant time interval color data from 
a motion picture; and maximum movement object ex- 
traction means for extracting, at a constant time interval, 
an object that travels a maximum distance. The linking 
means links the thus obtained motion vector, the color 
data and the object that moves the maximum distance 
to the segments of the graphic representation that cor- 
respond to the overall length of the motion picture. The 
clip length ratio display means displays the items in the 
graphic segments that are linked together. 
[0017] This display device can employ the above de- 
scribed clip display method. 

[0018] According to a first aspect of the invention, a 
clip display method, whereby clips obtained by dividing 
a motion picture are displayed using typical images of 
the clips, comprises the steps of 

dividing a graphic representation of the overall 
length of the motion picture into segments at ratios 
of lengths of the clips to the overall length of the 
motion picture; and 

linking the segments of the graphic representation 
with the typical images to display the clips. The lo- 
cations of the clips in the motion picture and the ratio 
of the length of each clip to the overall length of the 
motion picture can be visually ascertained. 

[0019] Optionally, according to the invention, the seg- 
ments of the graphic representation are linked with the 
typical images by using colors or numerals. Since a seg- 
ment having a color or a numeral provided for the typical 
image is searched for in the graphic representation of 
the overall length of the motion picture, the positioning 
in the motion picture of the clips represented by the typ- 
ical images can be understood at a glance. 
[0020] According to a second aspect of the invention, 
a clip display method, whereby clips obtained by divid- 
ing a motion picture are displayed using typical images 
of the clips, comprises the steps of 



dividing a graphic representation of the overall 
length of the motion picture into segments equal in 
number to the clips to display the clips with the typ- 
ical images; and 
s displaying, in each of the segments, a change in an 
image color for a corresponding clip. The change in 
the color of the typical image for each clip can be 
understood., 

w [0022] According to a fourth aspect of the invention, 
a clip display method, whereby clips obtained by divid- 
ing a motion picture are displayed using typical images 
of the clips, comprises the steps of 
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dividing a graphic representation of the overall 
length of the motion picture into segments equal in 
number to the clips to display the clips with the typ- 
ical images; and 

displaying, between the segments of the graphic, 
border lines that are shaped in consonance with a 
numerical differential value between the images for 
the clips corresponding to the segments. Whether 
the differential value between the scenes of the pre- 
ceding and the succeeding clip at a border line is 
great or small can be determined from the shape of 
the border line. 



[0023] Optionally according to the invention, the width 
or the type of line used for the border lines is changed 
30 in consonance with the numerical differential value. It 
can be ascertained that there is a large change of scene 
when a border line is thick, and that there is a small 
change of scene when a border line is thin. In addition, 
it can be ascertained that there is a large change of 
3$ scene when a border line is represented by a solid line, 
and that there is a small change of scene when a border 
line is represented by a broken line. 
[0024] According to a fifth aspect of the invention, a 
clip display method, 
40 whereby clips obtained by dividing a motion picture are 
displayed using typical images of the clips, comprises 
the steps of 



dividing a graphic representation of the overall 
length of the motion picture into segments equal in 45 
number to the clips so as to display the clips with 
the typical images; and 

displaying a motion vector, which represents the 
movement of an image in each of the clips, in a cor- 
responding segment of the graphic representation, so 
The movement of the typical image of the clip can 
be identified by monitoring the motion vector 



[0021] According to a third aspect of the invention, a 
clip display method, whereby clips obtained by dividing 
a motion picture are displayed using typical images of 
the clips, comprises the steps of 



55 



dividing a graphic representing the overall length of 
the motion picture into segments equal in number 
to the clips to display the clips with the typical im- 
ages; and 

extracting an object that travels the greatest dis- 
tance from an image for each of the clips and dis- 
playing the object in a corresponding segment of 
the graphic representation. Therefore, what is mov- 
ing in a typical image can be understood at a glance. 

[0025] According to a sixth aspect of the invention, a 
clip display method, 

whereby clips obtained by dividing a motion picture are 
displayed using typical images of the clips, comprises 
the steps of 
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dividing a graphic representation of the overall 
length of the motion picture into segments equal in 
number to the clips to display the clips with the typ- 
ical images; and 

displaying, in the segments of the graphic represen- 
tation, types of sounds that are recorded together 
with images for corresponding clips. Since the 
sounds recorded with the images of the clips are 
represented as either voices or music, the contents 
of a motion picture for a clip can be easily under- 
stood. 

[0026] Optionally according to the invention, when 
one typical image is selected, a mark is displayed in a 
corresponding segment of the graphic representation of 
the overall length of a motion picture, so that, when one 
location in the graphic representation of the overall 
length of the motion picture is selected, a corresponding 
typical image is displayed. The correspondence be- 
tween a segment of the graphic representation of the 
overall length of the motion picture and a clip that is rep- 
resented by a typical image can be ascertained at a 
glance. 

[0027] According to a seventh aspect of the invention, 
a clip display device, which displays clips obtained by 
dividing a motbn picture using typical images of the 
clips, comprises: index display means for displaying the 
typical images of clips; clip length ratio display means 
for dividing a graphic representation of the overall length 
of a motion picture into segments at ratios of the lengths 
of the clips to the overall length of the motion picture; 
and linking means for linking the typical images dis- 
played by the index display means to the segments of 
the graphic representation displayed by the clip length 
ratio display means. Segments of the graphic represen- 
tation of the overall length of the motion picture are dis- 
played that correspond to the ratios of the lengths of in- 
dividual clips that are represented by the typical images 
to the overall length of the motion picture. 
[0028] Optionally according to the invention, the link- 
ing means assigns the same color or numeral to the typ- 
ical image and the segment that are linked together. The 
segments of the graphic representation of the overall 
length of the motion picture are displayed in colors as- 
signed for corresponding typical images, or numerals 
assigned to the typical images are displayed in corre- 
sponding segments. 

[0029] Optionally according to the inventbn, further 
provided is motion vector calculation means for extract- 
ing two sequential frames from a motion picture at a con- 
stant time interval in order to calculate a motion vector. 
The linking means links the obtained motion vector with 
the segments of the graphic representation, and the clip 
length ratio display means displays the motion vector in 
the segments of the graphic representation. As a result, 
the clips can be displayed using the method of the sec- 
ond aspect. 

[0030] Optionally according to the invention, further 



provided is color data calculation means for extracting 
frames from a motion picture at a constant time interval 
in order to calculate data for the dominant color in each 
of the frames. The linking means links the obtained color 
s data to the segments of the graphic representation, and 
the clip length ratio display means displays colors rep- 
resented by the color data in the segments of the graphic 
representation. As a result, the clips can be displayed 
using the method of the third aspect. 
io [0031] Optionally according to the invention, further 
provided is differential value storage means for storing 
a differential value acquired between two sequential 
frames in order to divide a motion picture into clips. The 
linking means employs the differential value to desig- 
ns nate the shapes of border lines between the segments 
of the graphic representation. The clip length ratio dis- 
play means displays the border lines having the desig- 
nated shapes between the segments of the graphic rep- 
resentation. As a result, the clips can be displayed using 
20 the methods of the fourth aspects. 

[0032] Optionally according to the invention, further 
provided is maximum movement object extraction 
means for extracting two sequential frames from a mo- 
tion picture at a constant time interval and calculating a 
25 motion vector therefrom to acquire an object that travels 
a maximum distance. The linking means links the ob- 
tained objects with the segments of the graphic repre- 
sentation, and the clip length ratio display means dis- 
plays the objects in the segments of the graphic repre- 
30 sentation. As a result, the clips can be displayed using 
the method of the fifth aspect. 
[0033] Optionally according to the invention, further 
provided is sound data acquisition means for extracting 
sound that is recorded with images at a constant time 
35 interval and for identifying a sound type. The linking 
means links the sound type that is identified with the 
segments of the graphic representation, and the clip 
length ratio display means displays graphic symbols 
representing the sound types in the segments of the 
to graphic representation. As a result, the clips can be dis- 
played using the method of the sixth aspect. 
[0034] Optionally according to the invention, further 
provided are at least two of a motion vector calculation 
means for extracting at a constant time interval two se- 
45 quential frames from a motion picture to calculate a mo- 
tion vector, a color data calculation means for extracting 
at a constant time interval frames from a motion picture 
to calculate color data that are mainly included in the 
frames, a differential value storage means for storing a 
50 differential value between two sequential frames that is 
acquired to divide a motion picture into clips, a maximum 
movement object extraction means for extracting at a 
constant time interval two sequential frames from a mo- 
tion picture and calculating a motion vector therefrom to 
>5 acquire an object that travels the maximum distance, 
and a sound data acquisition means for extracting at a 
constant time interval sound that is recorded with a mo- 
tion picture and for identifying sound types; and a selec- 
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tion means for selecting one or more of those means. 
When the motion vector calculation means is selected, 
the linking means links the obtained motion vector with 
the segments of the graphic representation, and the clip 
length ratio display means displays the motion vector in 
the segments of the graphic representation. When the 
color data calculation means is selected, the linking 
means links the obtained color data with the segments 
of the graphic representation, and the dip length ratio 
display means displays colors indicated by the color da- 
ta in the segments of the graphic representation. When 
the differential value storage means is selected, the link- 
ing means employs the differential value to designate 
shapes of border lines between the segments of the 
graphic representation, and the clip length ratio display 
means displays the border lines having the designated 
shapes between the segments of the graphic represen- 
tation. When the maximum movement object extraction 
means is selected, the linking means links the obtained 
objects with the segments of the graphic representation, 
and the clip length ratio display means displays the ob- 
jects in the segments of the graphic representation. And 
when the sound data acquisition means is selected, the 
linking means links the sound type that is identified with 
the segments of the graphic representation, and the clip 
length ratio display means displays graphic symbols 
representing the sound types in the segments of the 
graphic representation. As a result, the clips can be dis- 
t played using two or more arbitrarily selected clipping 
methods of the first to sixth aspects. 
[0035] Optionally according to the invention, further 
provided are selection means for selecting a position in 
a typical image on or in the graphic representation of the 
overall length of a motion picture; and marking means 
for displaying a mark in a corresponding typical image 
when a segment of said graphic representation is se- 
lected, and for displaying a mark in a corresponding seg- 
ment of the graphic representation in order to identify a 
typical image when a corresponding image is selected. 
[0036] These and other aims and features of the 
present invention wilt become more readily apparent 
from the following detailed description of exemplary em- 
bodiments and the accompanying drawings in which : 

Fig. 1 is an explanatory diagram showing a clip dis- 
play method according to a first embodiment of the 
present invention; 

Fig. 2 is a diagram illustrating an editing screen of 
an editing device; 

Fig. 3 is a block diagram illustrating the arrange- 
ment of a clip display device according to the first 
embodiment of the present invention; 
Fig. 4 is an explanatory diagram showing a clip dis- 
play method according to a second embodiment of 
the present invention; 

Fig. 5 is a block diagram illustrating the arrange- 
ment of a clip display device according to the sec- 
ond embodiment of the present invention; 
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Fig. 6 is an explanatory diagram showing a clip dis- 
play method according to a third embodiment of the 
present invention; 

Fig. 7 is a block diagram illustrating the arrange- 
ment of a clip display device according to the third 
embodiment of the present invention; 
Fig. 8 is an explanatory diagram showing a clip dis- 
play method according to a fourth embodiment of 
the present invention; 

Fig. 9 is a block diagram illustrating the arrange- 
ment of a clip display device according to the fourth 
embodiment of the present invention; 
Fig. 1 0 is an explanatory diagram showing a clip dis- 
play method according to a fifth embodiment of the 
present invention; 

Fig. 11 is a block diagram illustrating the arrange- 
ment of a clip display device according to the fifth 
embodiment of the present invention; 
Fig. 1 2 is an explanatory diagram showing a clip dis- 
play method according to a sixth embodiment of the 
present invention; 

Fig. 13 is a block diagram illustrating the arrange- 
ment of a clip display device according to the sixth 
embodiment of the present invention; 
Fig. 1 4 is an explanatory diagram showing a clip dis- 
play method according to a seventh embodiment of 
the present invention; 

Fig. 15 is a block diagram illustrating the arrange- 
ment of a clip display device according to the sev- 
enth embodiment of the present invention; 
Fig. 1 6 is an explanatory diagram showing a clip dis- 
play method according to an eighth embodiment of 
the present invention 

Fig. 17 is a block diagram illustrating the arrange- 
ment of a clip display device according to the eighth 
embodiment of the present invention; 
Fig. 1 8 is an explanatory diagram showing a con- 
ventional clip display method; 
Fig. 1 9 is an explanatory diagram showing another 
example of a conventional clip display method; and 
Fig. 20 is a block diagram illustrating the arrange- 
ment of a conventional clip display device. 



[0037] The embodiments of the present invention will 
« now be described referring to the accompanying draw- 
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(First Embodiment) 

so [0038] An explanation will be given for a clip display 
for a first embodiment where the locations of clips in a 
motion picture and the ratio of the length of each clip to 
the length of a motion picture are displayed to be dis- 
cernible. 

55 [0039] First, an operating screen of a motion picture 
editing device that was developed by a group including 
the present inventor will be described. As is shown in 
Fig. 2, on this screen are displayed an operating button 



5 



9 



EP 0 907 147 A2 



10 



21, for fetching digital images from a file; an operating 
button 22, for fetching images from a digital video cam- 
era; a pre-edited index display 23, for displaying indexes 
for pre-edited clips; a bar 24, for displaying relationships 
between individual clips and a motion picture; and a s 
post-edited index display 25, for displaying an index for 
a clip, the scene of which has been selected by an editor. 
[0040] When the editor manipulates the button 21 on 
the operating screen, digital images are read from a file; 
and when the editor manipulates the button 22, digital 10 
video image are read from a video camera. The images 
are divided into clips for individual scenes, and the in- 
dexes for the clips are displayed on the pre-edited index 
display 23. The relationship between each clip and the 
motion picture is displayed in the bar 24. is 
[0041] When the editor selects clips to be edited from 
the indexes that are displayed on the pre-edited index 
display 23, while referring to.the display on the bar 24, 
the indexes for the selected clips are displayed on the 
post-edited index display 25, and the clips are linked in 20 
the order at which they were selected. The editing de- 
vice is described in detail in another patent application. 
[0042] According to the clip display method of this in- 
vention, the display on the pre-edited index display 23 
is linked with the display on the bar 24 so that the editor & 
can easily ascertain the contents of the clips. According 
to the clip display method of the first embodiment, as is 
shown in Fig. 1, various colors corresponding to the in- 
dexes for the clips on the pre-edited index display 23 
are used to establish the lengths of the individual dips 30 
that are displayed on the bar 24. Therefore, the ratio of 
the length of each dip to the motion picture and the lo- 
cation of each dip in the motion picture can be ascer- 
tained at a glance. 

[0043] The display device for providing such a display 35 
comprises, as is shown in Fig. 3, video reading means 

1 , for reading digital images from a video camera or from 
a hard disk; clip division means 2, for dividing a digital 
image into clips; clip division storage means 4, for hold- 
ing IN point and OUT point data for the clips; index gen- *o 
eration means 3, for generating indexes for the clips; 
index display means 6, for displaying the indexes gen- 
erated by the index generation means 3; clip length ratio 
calculation means 5, for calculating a ratio for the length 

of each clip; clip length ratio display means 8, for dis- 45 
playing on the bar 24 separate segments whose sizes 
correspond to the length ratio for individual clips; and 
display means linking means 7, for, linking the indexes 
displayed on the index display means 6 with the sepa- 
rate segments on the bar 24. so 
[0044] The video reading means 1 of the display de- 
vice reads from a storage device, such as a hard disk, 
a video signal that is encoded using the DV method and 
transmits the encoded signal to the clip division means 

2. 55 
[0045] The DV method is based on the specifications 

for digital video devices, such as digital video cameras, 
and conforms to the video cassette recorder specifica- 



tions that were established in 1996, •Specifications for 
Consumer-Use Digital VCRs (HD Digital VCR Confer- 
ence, 1996).' DCT (dispersion cosine conversion) is 
performed for each block of DV data, and image com- 
pression is provided by reducing the spatial redundancy 
in a frame for this process. One macro btock for DV data 
consists of four horizontally arranged luminance signal 
blocks and two chrominance signal (CR and CB) blocks 
that are located at corresponding positions, and has a 
color component form of 4:1 : 1 . A super block is consti- 
tuted by 27 of the macro blocks, and a 5 x 10 array of 
the super blocks constitutes one frame. 
[0046] When the clip division means 2 receives a vid- 
eo signal, it generates clips using a method selected by 
an operator, i.e., a method for dividing an image at spe- 
cific time intervals (e.g., intervals of one second) or an 
image division method for automatically detecting a 
change of scene. When the second method is selected, 
a differential value between the video data for two se- 
quentially positioned frames is calculated (for DV data, 
the compression rate for each macro block is employed 
as the video data for the calculation, and the sum of the 
differential values between frames is acquired). When 
the differential value is greater than a threshold value, 
it is determined that at that position there has been a 
change of scene. 

[0047] The index generation means 3 generates an 
index using the first ' frame of a clip obtained by the clip 
division means 2. For DV, for example, a Bitmap that is 
so generated by extracting a direct current element is 
regarded as an index. 

[0048] The index display means 6 displays in order 
the Bitmaps generated along the time axis by the index 
generation means 3. 

[0049] The clip division data storage means 4 stores 
IN point and OUT point data for the clips provided by the 
clip division means 2, and the clip length ratio calculation 
means 5 employs the IN point and OUT point data for 
clip division to calculate the ratio of the length of each 
clip to the overall length of the motion picture. The clip 
length ratio display means 8 employs the length ratios 
obtained by the clip length ratio calculation means 5 to 
divide the area of the bar 24, so that the ratio of the 
length of each clip to the total length of a motion picture 
is readily apparent. 

[0050] The display means linking means 7 links the 
indexes displayed by the index display means 6 with the 
segments on the bar 24 displayed by the clip length ratio 
display means 8. For example, an V is displayed in the 
segment on the bar 24 that corresponds to the index for 
clip x, or n colors are assigned in order to the clips and 
a color corresponding to a clip is assigned to a corre- 
sponding segment on the bar 24. 
[0051] Therefore, when an editor looks at the indexes 
and the bar that are displayed, he or she can understand 
at a glance the ratio of the length of each clip to the over- 
all length of the motion picture and the location of each 
clip in the motion picture, and can easily edit the images. 
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(Second Embodiment) 

[0052] For a second embodiment, an explanation will 
be given for a clip display where the movement of an 
image in each clip is displayed so as to be discernible, s 
[0053] According to this display method, as is shown 
in Fig. 4, the area of a bar 24 is divided into segments 
using the ratios of the lengths of individual clips to the 
overall length of a motion picture, and the movement of 
an image for a clip is displayed by using arrows in a cor- 10 
responding segment. When, for example, a person is 
displayed in the index of a clip, the direction in which the 
person is to be moved in the image can be understand 
at a glance by looking at the arrows in a corresponding 
segment of the bar 24. is 
[0054] A display device for providing such a display 
comprises, as is shown in Fig. 5, motion vector calcula- 
tion means 9, for determining the movement of an object 
from an image that is read. The other arrangements are 
the same as those for the device of the first embodiment 20 
in Fig. 3. 

[0055] The motion vector calculation means 9 ex- 
tracts two sequential frames from the video reading 
means 1 at a constant time interval (e.g., an interval of 
two seconds), and calculates a motion vector for prede- 2s 
termined segments (e.g., centers) of frames. The calcu- 
lation performed for the motion vector is the same as 
that performed for Mpegl and 2. When the motion vector 
calculation means 9 obtains the motion vector, it con- 
verts the vector into one that most nearly approximates 30 
one of a number of vectors for n directions (e.g., four 
directions). When no motion vector is obtained, it is as- 
sumed that no vector exists. 

[0056] The display means linking means 7 employs 
the vector obtained by the motion vector calculation 3S 
means 9 to perform the mapping of the bar 24 that is 
displayed by the clip length ratio display means 8. 
[0057] The arrows that represent motion vectors for 
the images of the individual clips are displayed on the 
bar 24. For a clip for which the ratio of its length to the *o 
overall length of a motion picture is large, a plurality of 
arrows are displayed that represent the motion vectors 
at constant time intervals. 

[0058] Therefore, when an editor looks at the displays 
of the indexes and the bar 24, he or she can visualize 
the movement of the image in each clip, and can easily 
select scenes for editing. 

(Third Embodiment) 

so 

[0059] For a third embodiment, an explanation will be 
given for a clip display where a change of colors in an 
image of each clip is displayed. 
[0060] According to this display method, as is shown 
in Fig. 6, the area of a bar 24 is divided into segments ss 
at ratios corresponding to the lengths of individual clips 
to the overall length of a motion picture, and a main color 
used in an image for a clip and the changing of the color 



are displayed in a corresponding segment. When, for 
example, the scene for a clip represents the sunset, fol- 
lowing the time axis, the color displayed in a correspond- 
ing segment on the bar 24 is changed to a darker color. 
Thus, how the image color for the clip is changed can 
be understood. 

[0061] A display device for providing such a display 
comprises, as is shown in Fig. 4, color data acquisition 
means 10 for acquiring color data from an image that is 
read. The other arrangements are the same as those 
for the device of the first embodiment shown in Fig. 3. 
[0062] The color data acquisition means 10 acquires 
a single frame from the video reading means 1 at con- 
stant time intervals, and employs luminance and 
chrominance signals to acquire, for each block, color da- 
ta that are included in the frame. Then, the color data 
for each block is changed to one that most nearly ap- 
proximates one color of a set of n colors. The color of 
the set of n colors that is most used in the frame is de- 
termined and is employed as the model color for the 
frame. While in this embodiment only one color is used 
as the model color for a frame, a frame may be divided 
into a plurality (m) of regions, a model color for each of 
the regions may be determined, and the so obtained m 
colors may be employed as model colors for the frame. 
[0063] The display means linking means 7 employs 
the model color obtained by the cobr data calculation 
means 10 to perform the mapping for the bar 24 that is 
displayed by the clip length ratio display means 8. 
[0064] The model colors for images in the individual 
clips are displayed on the bar 24. For clips for which the 
ratios of their lengths to the overall length of the motion 
picture are large, the model colors are displayed in order 
at constant time intervals. 

[0065] Therefore, when an editor looks at the displays 
of the indexes and the bar 24, he or she can grasp the 
meaning of the changing colors for the image in each 
clip, and can easily select scenes for editing. 

(Fourth Embodiment) 

[0066] For a fourth embodiment, an explanation will 
be given for a clip display where the degree of change 
between scenes is displayed. 
[0067] According to this display method, as is shown 
in Fig. 8, the area of a bar 24 is divided into segments 
at ratios of the lengths of individual clips to the overall 
length of a motion picture, and the thickness of a border 
line that is displayed corresponds to the degree of 
change between scenes. When, for example, a scene 
for a specific clip differs completely from a scene for a 
succeeding clip, a thick border line is displayed. But 
when a scene for a specific clip is similar to a scene for 
a succeeding clip, a thin border line is , displayed. Thus, 
the degree of change between scenes can be derived 
from the relative thickness of a border line. 
[0068] A display device for providing such a display 
comprises, as is shown in Fig. 9, differential value stor- 
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age means 11 for holding the differential value calculat- 
ed by the clip division means 2 for the detection of 
scenes. The other arrangements are the same as those 
for the device of the first embodiment shown in Fig. 3. 
[0069] In this device, when the clip division means 2 
detects scene changes in images carried by a video sig- 
nal and divides the images into segments, the differen- 
tial values between the frames that are calculated for 
the clip division by the clip division means 2 are stored 
in the difference value storage means 1 1 . 
[0070] After the clip division means 2 divides the im- 
ages at constant time intervals, for each division point it 
calculates a differential value between the preceding 
and the succeeding frame and stores it in the differential 
value storage means 11 . 

[0071] The display means linking means 7 employs 
the differential value data stored in the differential value 
storage means 11 to sort into n groups the image sep- 
aration types for the clips, and transmits a message to 
that effect to the clip length ratio display means 8. Based 
on the received data, the clip length ratio display means 
8 displays the bar 24, while changing the thicknesses of 
the border lines on the bar 24. 
[0072] In this manner, border lines having widths that 
correspond to the degree of the changes between 
scenes are displayed in the bar 24. Instead of changing 
the thickness of the border lines, the clip length ratio dis- 
play means 8 may change the types of border lines that 
, are used (broken lines or wavy lines) or the colors of the 
border lines, 

[0073] Therefore, when an editor looks at the displays 
of the indexes and the bar 24, he or she can understand 
how the images change between clips, and can easily 
select scenes for editing. 

(Fifth Embodiment) 

[0074] For a fifth embodiment, an explanation will be 
given for a clip display in which, for each clip, the object 
in the image that moves the most is displayed. 
[0075] According to this display method, as is shown 
in Fig. 1 0, the area of the bar 24 is divided into segments 
at ratios of the lengths of individual clips to the overall 
length of a motion picture, and the object that moves the 
most is extracted for each clip and is displayed in a cor- 
responding segment. When two planes are displayed in 
a segment on the bar 24, it means that the planes are 
moving in the index for a corresponding clip and that the 
moving planes also appear in an image after a specific 
time has elapsed. When a horse is displayed only on 
the right side of a segment on the bar 24, it means that 
no moving object is shown in the index, but that a moving 
horse appears in an image after a predetermined time 
has elapsed. 

[0076] A display device for providing such a display 
comprises, as is shown in Fig. 11 , maximum movement 
object extraction means 12 for detecting, from images 
that is read, the object that has the maximum move- 
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ment. The other arrangements are the same as those 
for the device of the first embodiment shown in Fig. 3. 
[0077] The maximum movement object extraction 
means 1 2 extracts two sequential frames from the video 
s reading means 1 at a constant time interval, calculates 
a motion vector for each macro block, and cuts out an 
image portion that reflects the maximum motion vector 
and that moves in the same way. 
[0078] The display means linking means 7 performs 
10 mapping, on the bar displayed by the clip length ratio 
display means 8, for an image that is extracted by the 
maximum movement object extraction means 12. 
[0079] As a result, for each clip, an object that moves 
the most in an image is displayed on the bar 24. For a 
'5 clip for which the ratio of its length to the overall length 
of a motion picture is large, a plurality of objects that 
move the most are selected from among the images ex- 
tracted at constant time intervals along the time axis and 
are displayed. 

20 [0080] Therefore, when an editor looks at the displays 
of the indexes and the bar 24, he or she can identify for 
each clip an object that is moving in an image, and can 
easily select scenes for editing. 

25 (Sixth Embodiment) 

[0081] For a sixth embodiment, an explanation will be 
given for a dip display where symbols are displayed for 
the types of sounds recorded with images. 

30 [0082] According to this display method, as is shown 
in Fig. 12, the area of the bar 24 is divided into segments 
at ratios of the lengths of individual clips to the overall 
length of a motion picture. When the sound recorded 
with images for a clip is a voice, the figure of a person 

35 is displayed in a segment, and when the sound recorded 
with images is music, a note is displayed. When both 
voice and music are recorded, the figure of a person and 
a note are overlapped on a display. 
[0083] A display device for providing such a display 

40 comprises, as is shown in Fig. 13, sound data acquisi- 
tion means 1 3 for acquiring sounds recorded with imag- 
es and identifying types of the sounds. The other ar- 
rangements are the same as those of the device for the 
first embodiment in Fig. 3. 

45 [0084] The sound data acquisition means 1 3 extracts 
sound data, which have been recorded with images, 
from the video reading means 1 at constant time inter- 
vals, and examines the frequency bands for sounds and 
the management of changes in the frequencies, and the 

50 presence/absence of a soundless portion to identify 
whether the sound is a voice or is music (the voice of a 
person includes an invoiced portion, while music does 
not include such a portion). 

[0085] The display means linking means 7 transmits 
55 the sound data obtained by the sound data acquisition 
means 13 to the clip length ratio display means 8, which, 
based on the sound data, displays the figure of a person, 
or a note, in each segment on the bar 24. 
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[0086] In this manner, the figures of persons or notes 
are displayed to represent the sound type, and for a clip 
for which the ratio of its length to the overall length of a 
motion picture is large, a plurality of sound types are 
displayed that are identified at constant time intervals 
along the time axis. 

[0087] Therefore, when an editor looks at the displays 
of the indexes and the bar 24, he or she can ascertain 
what types of sounds are recorded with the images for 
clips, and can easily select scenes for editing. 

(Seventh Embodiment) 

[0088] A display method according to a seventh em- 
bodiment can be provided by the arbitrary combination 
of the display methods of the second to the sixth em- 
bodiments. 

[0089] In Fig. 1 4 is shown a combination of the display 
methods of the second and the fifth embodiments. With 
this display, an object in the index that is moving and the 
direction of its movement can be ascertained. 
[0090] A display device for providing such a display 
comprises, as is shown in Fig. 15, a means unit 14 that 
includes motion vector calculation means, color data 
calculation means, differential value storage means, 
maximum movement object extraction means and 
sound data acquisition means; and display data selec- 
tion means 15 for selecting several means from the 
means unit 14. The other arrangements are the same 
as those for the device according to the first embodiment 
in Fig. 3. 

[0091] With this device, when an editor manipulates 
the display data selection means 15 to select one or 
more means from the means unit 14, the selected 
means is activated to perform a display method selected 
from those of the second to the sixth embodiments. 
[0092] When a combination of the clip display meth- 
ods is employed, an editor can obtain a better under- 
standing of the contents of a clip, and as the editor can 
efficiently locate a desired scene, the editing is easy. 

(Eighth Embodiment) 

[0093] In an eighth embodiment, an explanation will 
be given for a clip display where the correspondence 
between segments displayed on a bar and clips is so 
depicted that it can be easily understood. 
[0094] According to this display method, as is shown 
in Fig. 16, the area of a bar 24 is divided into segments 
at ratios of the lengths of individual clips to the overall 
length of a motion picture. When an editor selects the 
index for a specific clip, a triangular mark is displayed 
in a corresponding segment on the bar 24. And when 
an editor selects one segment on the bar 24, a mark is 
displayed in the index of a corresponding clip. There- 
fore, the editor can immediately understand the corre- 
spondence between the clips and the segments on the 
bar 24, and can quickly find a specific clip by designating 



an approximate location of the motion picture on the bar 
24. 

[0095] A display device for providing such a display 
comprises, as is shown in Fig. 17, segment selection 

s means 16 for permitting an editor to select a segment, 
and marking means 1 7 for providing a mark at a selected 
segment and at its linking destination. The other ar- 
rangements are the same as those of the device accord- 
ing to the first embodiment in Fig. 3. 

10 [0096] In this device, when an editor selects one index 
using the segment selection means 16, the selected lo- 
cation is transmitted to the display means linking means 
7. 

[0097] The marking means 1 7 designates a mark fig- 
'5 ure that is to be provided for a selected index, and a 
mark figure that is to be provided for a linking position 
in the bar 24. The display means linking means 7 trans- 
mits to the index display means 6 data that indicate the 
selected index and the mark figure that is to be provided 
20 for the index, and also transmits to the clip length ratio 
display means 8 the position of the bar 24 for linking with 
the selected index and the mark figure that is to be pro- 
vided for that position. 

[0098] Upon receipt of the data, the clip length ratio 

25 display means 8 displays a triangular mark at the select- 
ed position on the bar 24, and the index display means 
6 displays a shadow mark at the selected index. 
[0099] When the editor selects the position on the bar 
24 using the segment selection means 16, the same 

30 process is performed, and a shadow mark is displayed 
at the index to be linked with, while a triangular mark is 
displayed at the selected position on the bar 24. 
[01 00] Therefore, the editor can easily understand the 
correspondence of the clips and the segments on the 

35 bar 24 simply by manipulating the segment selection 
means 16, and editing can be easily performed. 
[0101] As is apparent from the above description, ac- 
cording to the display method of the present invention, 
the position of each clip relative to a motion picture, and 

40 the ratio of the length of each clip to the overall length 
of the motion picture can be visually identified. 
[01 02] According to the display method for displaying 
a motion vector, the movement of a typical image for a 
clip is ascertained; according to the display method for 

45 displaying a change in colors, the change in the image 
color for a clip is ascertained; and according to the dis- 
play method for changing the shape of border lines in 
consonance with the differential value between the im- 
ages, the magnitude of the change between scenes is 

50 ascertained from the thickness and the type of border 
line. And according to the display method for displaying 
an object that performs the maximum move, what is 
moving in a typical image is ascertained, and according 
to the display method for displaying a sound type, the 

55 types of sounds recorded with images is ascertained. 
[0103] Therefore, from these displays, an editor can 
exactly recall the contents of the individual clips of a mo- 
tion picture the editor took, or of a motion picture that 
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the editor previewed, and can precisely and efficiently 
perform editing. 

[0104] Further, according to the display method 
whereby when a typical image is selected, a mark is dis- 
played on a figure that represents the overall length of 
a motion picture and whereby when a specific position 
is selected in a figure, a corresponding typical image is 
displayed so that an editor can identify it, the corre- 
spondence between the area of the graphic represen- 
tation that represents the overall length of a moving im- 
age and the clip that is represented by a typical image 
can be understood at a glance. Further, when an ap- 
proximate position in the motion picture is selected by 
referring to the figure, a pertinent clip can be found. 
[01 05] The clip display device of the present invention 
can perform these clip display methods. 
[0106] Furthermore, in the display device that can 
perform a combination of these clip display methods, 
when a plurality of display methods are performed, the 
contents of a clip can be understood better, and the ef- 
ficiency of the editing can be further improved. 
[0107] While the present invention has been de- 
scribed with reference to preferred embodiments, vari- 
ous variations and modifications may be made without 
departing from the scope of the invention. 



Claims 

1. A clip display method, whereby clips obtained by 
dividing a motion picture are displayed using typical 
images of said clips, comprising the steps of: 

dividing a graphic representation of the overall 
length of said motion picture into segments at 
ratios of lengths of said clips to the overall 
length of said motion picture; and 
linking said segments of said graphic represen- 
tation with said typical images to display said 
clips. 

2. A clip display method according to claim 1 , wherein 
said segments of said graphic representation are 
linked with said typical images by using colors or 
numerals. 

3. A clip display method, whereby clips obtained by 
dividing a motion picture are displayed using typical 
images of said clips, comprising the steps of: 

dividing a graphic representation of the overall 
length of said motion picture into segments 
equal in number to said clips so as to display 
said clips with said typical images; and 
displaying a motion vector, which represents 
the movement of an image in each of said clips, 
in a corresponding segment of said graphic rep- 
resentation. 



A clip display method, whereby clips obtained by 
dividing a motion picture are displayed using typical 
images of said clips, comprising the steps of: 

dividing a graphic representation of the overall 
length of said motion picture into segments 
equal in number to said clips to display said 
clips with said typical images; and 
displaying, in each of said segments, a change 
in an image color for a corresponding clip. 

A clip display method, whereby clips obtained by 
dividing a motion picture are displayed using typical 
images of said clips, comprising the steps of: 

dividing a graphic representation of the overall 
length of said motion picture into segments 
equal in number to said clips to display said 
clips with said typical images; and 
displaying, between said segments of said 
graphic, border lines that are shaped in conso- 
nance with a differential value between said im- 
ages for, said clips corresponding to said seg- 
ments. 

6. A clip display method according to claim 5, wherein 
the width or the type of line used for said border lines 
is changed in consonance with said differential val- 
ue. 

7. A clip display method, whereby clips obtained by 
dividing a motion picture are displayed using typical 
images of said clips, comprising the steps of: 

dividing a graphic representing the overall 
length of said motion picture into segments 
equal in number to said clips to display said 
clips with said typical images; and 
extracting an object that travels the greatest 
distance from an image for each of said clips 
and displaying said object in a corresponding 
segment of said graphic representation. 

8. A clip display method, whereby clips obtained by 
dividing a motion picture are displayed using typical 
images of said clips, comprising the steps of: 

dividing a graphic representation of the overall 
length of said motion picture into segments 
equal in number to said clips to display said 
clips with said typical images; and 
displaying, in said segments of said graphic 
representation, types of sounds that are record- 
ed together with images for corresponding 
55 clips. 

9. A clip display method according to one of claims 1 
to 8, wherein, when one typical image is selected, 
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a mark is displayed in a corresponding segment of 
said graphic representation of the overall length of 
a motion picture, so that, when one location in said 
graphic representation of the overall length of said 
motion picture is selected, a corresponding typical s 
image is displayed. 

10. A clip display device, which displays clips obtained 
by dividing a motion picture using typical images of 
said clips, comprising: w 

index display means for displaying said typical 
images of clips; 

clip length ratio display means for dividing a 
graphic representation of the overall length of is 
a motion picture into segments at ratios of the 
lengths of said clips to said overall length of 
said motion picture; and 
linking means for linking said typical images 
displayed by said index display means to said 20 
segments of said graphic representation dis- 
played by said clip length ratio display means. 

11. A clip display device according to claim 10, wherein 
said linking means assigns the same color or nu- 2s 
meral to said typical image and said segment that 
are linked together. 

12. A clip display device according to claim 10 or 13, 
further comprising: 30 

motion vector calculation means for extracting 
two sequential frames from a motion picture at 
a constant time interval in order to calculate a 
motion vector, 35 
wherein said linking means links said obtained 
motion vector with said segments of said graph- 
ic representation, and 

wherein said clip length ratio display means dis- 
plays said motion vector in said segments of 40 
said graphic representation. 

13. A clip display device according to claim 10, 11 or 
12, further comprising: 

45 

color data calculation means for extracting 
frames from a motion picture at a constant time 
interval in order to calculate data for the domi- 
nant color in each of said frames, 
wherein said linking means links said obtained so 
color data to said segments of said graphic rep- 
resentation, and 

wherein said clip length ratio display means dis- 
plays colors represented by said color data in 
said segments of said graphic representation, ss 

14. A clip display device according to claim 10, 11, 12 
or 1 3, further comprising: 



differential value storage means for storing a 
differential value acquired between two se- 
quential frames in order to divide a motion pic- 
ture into clips, 

wherein said linking means employs said differ- 
ential value to designate the shapes of border 
lines between said segments of said graphic 
representation, and 

wherein said clip length ratio display means dis- 
plays said border lines having the designated 
shapes between said segments of said graphic 
representation. 

1 5. A clip display device according to any one of claims 
10 to 14, further comprising: 

maximum movement object extraction means 
for extracting two sequential frames from a mo- 
tion picture at a constant time interval and cal- 
culating a motion vector therefrom to acquire 
an object that travels a maximum distance, 
wherein said linking means links said obtained 
objects with said segments of said graphic rep- 
resentation, and 

wherein said clip length ratio display means dis- 
plays said objects in said segments of said 
graphic representation. 

1 6. A clip display device according to any one of claims 
10 to 15, further comprising: 

sound data acquisition means for extracting 
sound that is recorded with a motion picture at 
a constant time interval and for identifying a 
sound type, 

wherein said linking means links said sound 
type that is identified with said segments of said 
graphic representation, and 
wherein said clip length ratio display means dis- 
plays graphic symbols representing said sound 
types in said segments of said graphic repre- 
sentation. 

1 7. A clip display device according to any one of claims 
10 to 16, further comprising: 

at least two of a motion vector calculation 
means for extracting at a constant time interval 
two sequential frames from a motion picture to 
calculate a motion vector, a color data calcula- 
tion means for extracting at a constant time in- 
terval frames from a motion picture to calculate 
color data that are mainly included in said 
frames, a differential value storage means for 
storing a differential value between two se- 
quential frames that is acquired to divide a mo- 
tion picture into clips, a maximum movement 
object extraction means for extracting at a con- 
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stant time interval two sequential frames from 
a motion picture and calculating a motion vector 
therefrom to acquire an object that travels the 
maximum distance, and a sound data acquisi- 
tion means for extracting at a constant time in- s 
terval sound that is recorded with images and 
for identifying sound types; and 
a selection means for selecting one or more of 
those means, 

wherein when said motion vector calculation 10 
means is selected, said linking means links said 
obtained motion vector with said segments of 
said graphic representation, and said clip 
length ratio display means displays said motion 
vector in said segments of said graphic repre- *5 
sentation, 

wherein when said color data calculation 
means is selected, said linking means links said 
obtained color data with said segments of said 
graphic representation, and said clip length ra- 20 
tio display means displays colors indicated by 
said color data in said segments of said graphic 
representation, 

wherein when said differential value storage 
means is selected, said linking means employs 2s 
said differential value to designate shapes of 
border lines between said segments of said 
graphic representation, and said clip length ra- 
tio display means displays said border lines 
having the designated shapes between said 30 
segments of said graphic representation, 
wherein when said maximum movement object 
extraction means is selected, said linking 
means links said obtained objects with said 
segments of said graphic representation, and 3S 
said clip length ratio display means displays 
said objects in said segments of said graphic 
representation, and 

wherein when said sound data acquisition 
means is selected, said linking means links said 40 
sound type that is identified with said segments 
of said graphic representation, and said clip 
length ratio display means displays graphic 
symbols representing said sound types in said 
segments of said graphic representation. 45 

1 8. A clip display device according to any one of claims 
10 to 17, further comprising: 

selection means for selecting a position in a so 
typical image on or in said graphic representa- 
tion of the overall length of a motion picture; and 
marking means for displaying a mark in a cor- 
responding typical image when a segment of 
said graphic representation is selected, and for ss 
displaying a mark in a corresponding segment 
of the graphic representation in order to identify 
a typical image when a corresponding image is 



selected. 

1 9. A clip display device according to any one of claims 
10 to 18, further comprising: 

selection means for selecting a position in a 
typical image on or in said graphic representa- 
tion of the overall length of a mot ion picture; and 
marking means for displaying a mark in a cor- 
responding typical image when a segment of 
said graphic representation is selected, and for 
displaying a mark in a corresponding segment 
of the graphic representation in order to identify 
a typical image when a corresponding image is 
selected. 
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□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

£1 BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENGE(S) OR EXHffiIT(S) SUBMITTED ARE POOR QUALITY 

• • □ OTHER; ■ . •■■ _ . ______ . ' ' \ ' ■ : ; . ■ 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



